The Turing Test in Practice - 1950-2001

A Good Sense of Humour

The humour in Turing's original 1950 paper (see this Scrapbook page) is alive and well in the various programs that have been running under Turing Test conditions.

There is serious point to be made about Turing's humour: he was making it very clear that what he meant by 'intelligence' was something that could not only play chess but make a joke, connecting with the real language of real life.

Serious Artificial Intelligence people take a dim view of these efforts as a distraction from systematic research, but on the other hand, one virtue of Turing's scenario is that it obliges us to keep in mind the goal of machines rivalling human 'intelligence' without ifs or buts. And the public interest in these efforts has been a large factor in keeping Alan Turing alive and well in the public mind.

Sex on the brain...

Turing suggested that within 50 years a computer would pass a (actually not very stringent) comparison test, and now 2000 has come and gone. In 1991 the call went out for entries to a contest under Turing Test conditions. The Loebner Prize Contest has continued each year. In November 1991 the winning program was by Joseph Weintraub on the topic romantic conversation, and he was the winner again in 1992 and 1993. In 1994 the Loebner Prize Winner was Thomas Whalen. The topic of this computer program was Sex.

You can talk to the 1994 winning program by going to this page and opening the TELNET dialogue box. I asked 'How do men have sex with each other,' and was impressed that it could interpret this and give a suitably PC answer about gay sex. Saying I was a gay teenager, I asked how to find a boyfriend and got an all-purpose answer including 'Go to church.' I said, 'But churches are anti-gay,' and it said 'I cannot answer that.' After that it kept on repeating itself, and I couldn't see how anyone could possibly take it for a human; it seemed more like a politician on television. But it was fun.

In the contest on 16 December 1995, Joseph Weintraub regained supremacy: look at the results of the contest. For the first time the programs entered were not limited to a subject. But as you will see from the transcript sex still dominated the conversation.

The 1996 contest was held on April 16, and was won by Jason Hutchens (who has a great sense of humour) with a conversation which set new levels of intelligent discourse.

Unfortunately HEX is no longer available for on-line chat but you can download the HEX program in various formats directly here, and read his description of how it works.

There's a lively discussion of these contests and dialogues by Charles Platt, one of the people who were claiming to be genuine humans in the 1994 competition.

The 1997 contest was held on 29 April, and was won by David Levy. The winning conversation got off to a hot start:

PROGRAM: Did you see that story on CNN last night about the lesbian couple who came out at a White House party on Sunday?
JUDGE05: NO. I just came in yesterday. I'm still kind of jet-lagged.
PROGRAM: Ellen Degeneres was one of them - she was kissing her lover...

On 11 January 1998 the competition was held in Sydney, Australia. The winner was Robby Garner, who won again in 1999. In this contest the proceedings were open to the cyberpublic on Internet Relay Chat (see below.)




The fifty years are up

The competition for 2000, the fiftieth anniversary of Turing's prediction, was held at Dartmouth College, New Hampshire.

The winner was A. L. I. C. E. by Richard Wallace. Chat to A. L. I. C. E. here or here.

But no program entered for the competition came close to deceiving the judges. So we can say that Alan Turing was wrong in his prediction.

The next competition, in October 2001, was held at the Science Museum, London. Richard Wallace was again the winner. See the page about the competition.

The 2002 competition was held in Atlanta. It was won by Kevin L. Copple with EllaZ. The description boasts that 'The content of the CIA World Factbook has been added to Ella's resources. Now when you ask where or about any country or capital, an HTML page with a map and flag, together with economic, political, and geographic info is displayed.' Not so much Artificial Intelligence as American Intelligence. This sort of mindless disgorging of 'information' is the antithesis of what Alan Turing had in mind when illustrating his ideas with wit and humour.

The contest for October 2003 is hosted by Surrey University.
See these BBC features on the winner, Jabberwock and the British entrant, Jabberwacky. 'If Jabberwacky did not quite pass the Turing Test this time round, it certainly made the judges laugh.'

Talk to Jabberwock here.




Eliza shrinks your computer

Until the Loebner Prize, the most famous program aimed at Turing Test conditions was the ELIZA imitation psychiatrist, written by Joe Weizenbaum who is a notable AI sceptic.

You can talk to Eliza here, and there is another on-line version with commentary here.

Another sceptic, Mark Humphrys, has a page about his ELIZA-type program.

His scepticism about Turing Tests is much like the view I put in my book: that you can't separate words from the rest of life. Nevertheless, his programme induced a dialogue more human than any other I've seen, when someone logged in and chatted away without realising he was talking to a computer.

WARNING: This gets very rude, leading up to are you a stupid homosexual and logout. I wonder what Alan Turing would have throught of THAT...




More on computer conversation

Chatterbot Central

Cybelle, a further quick-fire conversation program, greets you on Agentland which is also a gateway to other 'intelligent agents.' Nothing like true Artificial Intelligence exists — and the imagination of Brian Aldiss, Stanley Kubrick and Steven Spielberg doesn't have much to do with the real content of Turing's ideas — but the more modest ambitions of limited 'agents' can be tested on-line.

A page on good questions to put.

Another on-line conversation program.




Life Imitates Art

Communication through computer terminals, a science-fiction idea in 1950, has long been possible on the Internet.

The chatline experience, communicating through symbols alone, leaving out the physical cues we use, is interesting in itself, raising all sorts of questions about truth and reality (and intelligence, or lack of it). People find they can say things in these conditions that they would never say 'in real life', just as they can tell a machine things they would never tell a living soul. The experience is certainly explored a lot by people, first on Internet Relay Chat, and now on countless web-based chatrooms.

I'm sure Turing would have been a keen user, and it might have kept him off the streets...


but a webcam changes the picture...

Hugh Loebner draws another connection with real life: he has written to me that

It was Alan Turing's unfortunate experience, and his consequent suicide that has given me the courage to come out of the closet and admit my sexual preferences.
His preferences are not the same as Alan Turing's were, but Mr Loebner has put out a manifesto, and another page referring to the Turing story.